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Abstract. We prove that any C 2 complete, orientable, connected, stable area-stationary 
surface in the sub-Riemannian Hcisenberg group H 1 is either a Euclidean plane or con- 
gruent to the hyperbolic paraboloid t = xy. 



1. Introduction 

Minimal surfaces in Euclidean space are area-stationary, a condition which is equivalent, 
by the Euler-Lagrange equation, to have mean curvature zero. An important question for 
such a variational problem is the classification of global minimizers. Hence is natural to 
consider the second variation. Minimal surfaces with non-negative second variation of the 
area are called stable minimal surfaces. It is well-known that minimal graphs are stable 
minimal surfaces (in fact area-minimizing by a standard calibration argument). A complete 
minimal graph must be a plane by the classical Bernstein's Theorem [6]. Bernstein result 
was later extended by do Carmo and Peng [15], and Fischer-Colbrie and Schoen [21], who 
proved that a complete stable oriented minimal surface in R 3 must be a plane. The proof in 
p?T] follows from more general results for 3-manifolds of non-negative scalar curvature. Non 
existence of non-orientable complete stable minimal surfaces in R 3 has been proved by Ros 
[35]. 

A similar analysis of the variational properties of area-minimizing surfaces is also of great 
interest in some special spaces, such as the three-dimensional Heisenberg group H . This is 
the simplest model of a sub-Riemannian space and of a Carnot group. It is also the local 
model of any 3-dimensional pseudo-hcrmitian manifold. For background on H 1 we refer the 
reader to Section [2] and [5] . 

Area-stationary surfaces of class C 2 in H 1 are well understood. It is well-known [TUJ, [33J 
that, outside the singular set given by the points where the tangent plane is horizontal, such 
a surface is ruled by characteristic horizontal segments. Moreover, based on the description 
of the singular set for i-graphs of class C 2 given by Cheng, Hwang, Malchiodi and Yang 
[TO] , Ritore and Rosales [33J proved that a C 2 surface E immersed in H 1 is area-stationary 
if and only if its mean curvature is zero and the characteristic segments in E meet orthogo- 
nally the singular curves. A similar result was independently obtained for area-minimizing 
i-graphs by Cheng, Hwang, and Yang [11] . Furthermore, the classification of C 2 complete, 
connected, orientable, area-stationary surfaces with non-empty singular set was provided 
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in |33j : the only examples are, modulo congruence, non- vertical Euclidean planes, the hy- 
perbolic paraboloid t = xy, and the classical left-handed minimal helicoids. Though some 
results for complete area-stationary surfaces with empty singular set have been proved, see 
for example [321 Thm. 5.4], [S] and [33J Prop. 6.16], a detailed description of such surfaces 
seems far from being established. This provides an additional motivation for the study of 
second order minima of the area in H 1 . 

As in the Euclidean case, we define a stable area-stationary surface in H 1 as a C 2 area- 
stationary surface with non-negative second derivative of the area under compactly sup- 
ported variations. These surfaces have been considered in previous papers in connection 
with some Bernstein type problems in H . Let us describe some related works. 

In [TU], a classification of all the complete C 2 solutions to the minimal surfaces equation 
for i-graphs in H 1 is given. In [33], this classification was refined by showing that the only 
complete area-stationary i-graphs are Euclidean non-vertical planes or those congruent to 
the hyperbolic paraboloid t — xy. By means of a calibration argument it is also proved in 
[33] that they are all area-minimizing. 

In [13j and [1] the Bernstein problem for intrinsic graphs in H 1 was studied. The notion 
of intrinsic graph is the one used by Franchi, Serapioni and Serra Cassano in [23j . Geomet- 
rically, an intrinsic graph is a normal graph over some Euclidean vertical plane with respect 
to the left invariant Riemannian metric g in H 1 defined in Section [2] A C 1 intrinsic graph 
has empty singular set. Examples of C 2 complete area-stationary intrinsic graphs different 
from vertical Euclidean planes were found in |13| . So a natural question is to study com- 
plete area-minimizing intrinsic graphs. A remarkable difference with respect to the case of 
the i-graphs is the existence of complete C 2 area-stationary intrinsic graphs which are not 
area- minimizing, see [13]. In [4], Barone, Serra Cassano and Vittone classified complete C 2 
area-stationary intrinsic graphs. Then they computed the second variation formula of the 
area for such graphs to establish that the only stable ones are the Euclidean vertical planes. 
An interesting calibration argument, also given in [3], yields that the vertical planes are in 
fact area- minimizing surfaces in M . 

In the interesting paper [T5], it is proven that C 2 complete stable area-stationary Eu- 
clidean graphs with empty singular set must be vertical planes. This is done by showing 
that if such a graph is different from a vertical plane then it contains a particular example 
of unstable surfaces called strict graphical strips. From the geometrical point of view, a 
graphical strip is a C 2 surface given by the union of a family of horizontal lines L t passing 
through and filling a vertical segment so that the angle function of the horizontal projection 
of L t is a monotonic function. The graphical strip is strict if the angle function is strictly 
monotonic. If the angle function is constant we have a piece of a vertical plane. We would 
like to remark that there are examples of complete area-stationary surfaces with empty 
singular set which do not contain a graphical strip, such as the sub- Riemannian catenoids 
t 2 = X 2 (x 2 + y 2 — A 2 ), A ^ 0. Hence the main result in [15] does not apply to general 
surfaces. 

The following natural step is to consider complete stable surfaces in H 1 . In fact, all the 
aforementioned results leave open the existence of stable examples different from intrinsic 
graphs or Euclidean graphs with empty singular set. The purpose of the present paper is 
to classify complete stable area-stationary surfaces in H 1 with empty singular set or not. In 
Theorem 16.11 we prove the following result 

The only complete, orientable, connected, stable area- stationary surfaces in 
H 1 of class C 2 are the Euclidean planes and the surfaces congruent to the 
hyperbolic paraboloid t — xy. 
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In particular, this result provides the classification of all the complete C 2 orientable area- 
minimizing surfaces in H 1 . 

In order to prove Theorem 16.11 we compute the second derivative of the area for some 
compactly supported variations of a C 2 area-stationary surface E by means of Riemannian 
geodesies. In Theorem 13.71 variations of a portion E' of the regular part of E in the direc- 
tion of vN + wT, where N is the unit normal to E and T is the Reeb vector field in H 1 , 
will be considered. Here v, w are assumed to have compact support in E, but not on E'. 
Hence the boundary of E' is moving along the variation. In Proposition 13.111 variations in 
the direction of wT of a C 2 area-stationary surface E with singular curves of class C 3 will 
be taken. Here w has compact support near the singular curves, and it is constant along 
the characteristic curves of E. Both types of variations will be combined to produce global 
ones in Proposition 15.21 Second variation formulas of the area for variations supported in 
the regular set have appeared in several contexts. In |10) . such a formula was obtained for 
C 3 surfaces inside a 3-dimensional pseudo-hermitian manifold. In [3], a second variation 
formula was proved for variations by intrinsic graphs of class C 2 . In [12! , it is computed the 
second derivative of the area associated to a C 2 variation of a C 2 surface along Euclidean 
straight lines. 

Once we have the second variation formula we proceed into two steps. First we prove in 
Theorem l4.7l that a C 2 complete oriented stable area-stationary surface with empty singular 
set must be a vertical plane. In fact, for such a surface E, the second derivative of the area 
for a compactly supported variation as in Theorem 13.71 is given by 

Z(u, u) = — I uC(u), 

where u is the normal component of the variation, and C is the hypoelliptic operator on E 
given in (|3.44p . By analogy with the Riemannian situation |3I we refer to X as the index 
form associated to E and to C as the stability operator of E. In Proposition 13.121 we see 
that the stability condition for E implies that I(u, u) ^ for any u S Co(E) which is also 
C 1 along the characteristic lines. Then we choose the function u :— \Nh\, where N is the 
Riemannian unit normal to E for the left invariant Riemannian metric g on H 1 defined in 
Section [5J Nh is the horizontal projection of N, and the modulus is computed with respect 
to the metric g. We see in Proposition 14.61 that this function u satisfies 

C{u) ^ 0, 

and the inequality is strict in pieces of E which are not contained inside Euclidean vertical 
planes. In such a case we produce a compactly supported non-negative function v in E 
so that inequality I(v,v) < still holds. To construct the function v we use the Jacobi 
vector field on E associated to the family of horizontal straight lines ruling E and which 
is studied in Lemma [4.51 Observe that the function \Nh\ is associated to the variational 
vector field induced by the surfaces equidistant to E in the Carnot-Caratheodory distance, 
see pp. Hence, our construction of the test function v is, in spirit, similar to that in the 
Euclidean case, where the equivalent test function is u = 1. Using Fischer-Colbrie's results 
|20j , a stable minimal surface in IR 3 is conformally a compact Riemann surface minus a finite 
number of points, so that a logarithmic cut-off function v of it = 1 has compact support 
and yields instability unless the surface is a plane. We remark that the function \Nh\ was 
already used as a test function in [3], [T3] and [T5] . 

In the second step of the proof of Theorem 16.11 we consider a complete area-stationary 
surface E with non-empty singular set. From the classification in [33], we conclude that E 
must be a non-vertical plane, congruent to the hyperbolic paraboloid t = xy, or congruent 
to a left-handed helicoid, see Proposition 15 . II for a precise statement. The first two types of 
surfaces are i-graphs and then they are area- minimizing by a calibration argument [33] . For 
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the third type we will combine our second variation formulas in Theorem 13.71 and Propo- 
sition 13.111 to produce the stability inequality Q(u) ^ 0, where Q is the quadratic form 
defined in (|5.8p . The construction of appropriate test functions with Q(u) < will prove 
the instability of the helicoids. It is interesting to observe that Q(u) ^ for functions u 
with support in the regular part of the helicoids. 

In the Heisenberg groups H™, with n ^ 5, there is no counterpart to Theorem 14.71 as 
some examples have been constructed in [3] of complete area-minimizing intrinsic graphs 
different from Euclidean vertical hyperplanes. For n = 2,3,4 it is still unknown if similar 
examples can be obtained. 

We would like to mention that examples of area-minimizing surfaces in H 1 with low 
Euclidean regularity have been obtained in [TT], [3T], [53] and [3U]. Hence our results are 
optimal in the class of C 2 area-stationary surfaces. 

Finally, the techniques in this paper can be employed to prove classification results for 
complete stable area-stationary surfaces under a volume constraint in the first Heisenberg 
group [36j . and inside the sub-Riemannian three-sphere |27j . 

We have organized this paper as follows: the next section contains some background ma- 
terial in several subsections. In the third one we recall known facts about area-stationary 
surfaces and we compute second variation formulas for the area. The fourth and fifth sec- 
tions treat complete stable surfaces without and with singular points, respectively. In the 
sixth section we state and prove the main result. 

After the distribution of this paper we were informed by Prof. Nicola Garofalo that The- 
orem 14.71 was proven, for the case of embedded surfaces, by Danielli, Garofalo, Nhieu and 
Pauls in late 2006, QI]. 



2. Preliminaries 



In this section we gather some previous results that will be used throughout the paper. 
We have organized it in several parts. 

2.1. The Heisenberg group. The Heisenberg group H 1 is the Lie group (M 3 , *), where the 
product * is defined, for any pair of points [z, t], [z' , t'] £ ]R 3 = C x R, by 

[z,t] * [z',t'] := [z + z',t + t' + lm(zz')}, (z = x + iy). 

For p € M , the left translation by p is the diffcomorphism L p (q) = p * q. A basis of left 
invariant vector fields (i.e., invariant by any left translation) is given by 

dx + y dt' dy X dt' df 

The horizontal distribution TL in H 1 is the smooth planar distribution generated by X and 
Y, The horizontal projection of a tangent vector U onto H will be denoted by Uh- A vector 
field U is horizontal if U — Uh- 

We denote by [U, V] the Lie bracket of two C 1 vector fields U and V on H 1 . Note that 
[X,T] = [Y,T] = 0, while [X, Y] = -2T, so that U is a bracket-generating distribution. 
Moreover, by Frobenius theorem we have that T-L is nonintegrable. The vector fields X and 
Y generate the kernel of the (contact) 1-form lu := —ydx + xdy + dt. 

2.2. The left invariant metric. We shall consider on H 1 the Riemannian metric g — (■ , •) 
so that {X,Y,T} is an orthonormal basis at every point. The restriction of g to T-L coin- 
cides with the usual sub-Riemannian metric in H 1 . Let D be the Levi-Civita connection 
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associated to g. From Koszul formula and the Lie bracket relations we get 



D X X = 0, 


DyY = 0, 


D T T 


= 0, 


(2.1) D X Y = -T, 


D X T = Y, 


DyT 


= -x, 


D Y X = T, 


D T X = Y, 


D T Y 


= -X. 


For any tangent vector U on H 1 we 


define J(U) 


:= DuT. 


Then we have J{X) = Y, 



J(Y) = —X and J(T) = 0, so that J 2 = —Id when restricted to H. It is also clear that 

(2.2) (J(U),V) + (U,J(V)) = 0, 

for any pair of tangent vectors U and V. The involution J : % — > % together with the 1-form 
u = —ydx + xdy + dt, provides a pseudo-hermitian structure on H , see [71 Sect. 6.4]. 

Let R be the Riemannian curvature tensor of g defined for tangent vectors U, V, W by 
R(U, V)W = D V D V W - D V D V W + D [uy] W. 
From (|2.ip and the Lie bracket relations we can obtain the following identities 

R(X, Y)X= -3Y, R(X, Y)Y = 3X, R(X, Y) T = 0, 

(2.3) R{X,T)X = T, R(X,T)Y = 0, R(X,T)T = -X, 
R(Y,T)X = 0, R(Y,T)Y = T, R(Y, T)T = -Y. 

We denote by flic the Ricci curvature in (H 1 ,*?) defined, for any pair of tangent vectors U 
and V, as the trace of the map W R(U, W)V. These equalities can be checked by taking 
into account (12.3[) 

(2.4) Ric(X, Y) = 0, Ric(X, T) = 0, Ric(F, T) = 0, 
Ric(X, X) = -2, Ric(y, Y) = -2, Ric(T, T) = 2. 

2.3. Horizontal curves and Carnot-Caratheodory distance. Let 7 : / — > H 1 be a 

piecewise C 1 curve defined on a compact interval Jcl. The length of 7 is the usual Rie- 
mannian length £(7) := Jj 17(e)! de, where 7 is the tangent vector of 7. A horizontal curve 7 
in H 1 is a C 1 curve whose tangent vector always lies in the horizontal distribution. For two 
given points in H 1 we can find, by Chow's connectivity theorem [24l Sect. 1.2.B], a horizontal 
curve joining these points. The Carnot-Caratheodory distance d cc between two points in H 1 
is defined as the infimum of the length of horizontal curves joining the given points. The 
topology associated to d cc coincides with the usual topology in K 3 , see [51 Cor. 2.6]. 

2.4. Geodesies and Jacobi fields in (M 1 , (7). A geodesic in (H 1 ,^) is a C 2 curve 7 such 
that the covariant derivative of the tangent vector field 7 vanishes along 7. 

Let j(s) = (x(s),y(s),t(s)). Dots will indicate derivatives with respect to s. We write 
j = xX + yY+(t — xy + xy) T. Then 7 is a geodesic in (H 1 , g) if and only if 

x = 2 (7, T) y, 

y = -2(i,T)i, 
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Let A be the constant i — xy + xy = Oy, T). An easy integration shows that the geodesic with 
initial conditions (x(0), y(0), t(0)) = (xo,yo>io) an d (x(0), y(0), i(0)) = (A,B,C) is given by 

x(s) = x + As f(2Xs) + Bs g{2Xs), 

(2.5) y(s) =y -As g(2Xs) + Bs f(2Xs), 

t(s) =t + As + (A 2 + B 2 )s 2 h{2Xs) + (Ax + By )s g(2Xs) 
+ (Ay -Bx )sf(2Xs), 
where /, g and h are the real analytic functions 

f sin(x) f 1 - cos(x) ( x ~ sin(x) 

/(x) := < x , g{x) := < x ^ , tt(x) := < x 2 ■ 

[l, x = [0, x = [0, x = 

In particular, we have 

(2.6) exp p (sv) = p + sv, for pel 1 and v € Hp or v || T p , 

which is a horizontal or vertical straight line. Here exp p denotes the exponential map of 
(H 1 ,.g)atp. 

In the next result we construct Riemannian Jacobi fields associated to C 1 families of 
Riemannian geodesies. 

Lemma 2.1. Let a : I — > H 1 be a C 1 curve defined on some open interval / CI. For 
any C 1 vector field U along a we consider the map F : / x K — > H 1 given by F(e, s) :— 
exp a / e \(sU a ( E )). Then, the variational vector field V £ (s) :— (dF / de){e, s) is C°° along the 
geodesic 7 e (s) := F(e, s). As a consequence, [%, V e ] — and V e satisfies the Jacobi equation 

(2.7) V:' + R(j e ,V e )% = 0, 

where the prime ' denotes the covariant derivative along the geodesic j £ . Moreover, if j £ is 
a horizontal straight line, then 

(2-8) V" - 3(V £ , J(%)) J(%) + \%\ 2 (V e ,T) T = 0. 

Remark 2.2. The classical proofs in Riemannian geometry of [%, V £ ] = and the fact that 
V £ satisfies the Jacobi equation do not apply directly in our setting since we only suppose 
that F is a C 1 map. 

Proof of Lemma \2.1\ Let (xo(e), yo(£),to(e)) and (A(e),B(e),C(e)) be the Euclidean coor- 
dinates of a{e) and U a r e -\, respectively. By using the expression of the Riemannian geodesies 
in (|2.5[) . we see that the map F(e, s) can be written as 

x(e, s) = x (e) + A(s)s f(2X(e)s) + B(e)s g(2X{e)s), 

y(e, s) = y {e) - A(e)s g(2X{e)s) + B(e)s f{2X(e)s), 

t(s, s) = t (e) + X{e)s + {A 2 + B 2 ){e)s 2 h{2X(e)s) + (A(e)x (e) + B(e)y (s))s g{2X{e)s) 
+ (A(s)yo(s) ~ B(s)x (s))sf(2X(e)s) 1 

where A(e) := C(e) — A(e)yo(e) + B(e)xo(e). Observe that the functions x (e), yo(e), to(e), 
A(e), B(e), C(e) and A(e) are C 1 . A direct computation of (dF/de) (e, s) shows that V e (s) 
is C°° along the geodesic 7 E (s). 

On the other hand, we can check that for all k g N and any of the Euclidean components 
4>(e, s) of F(e, s), the partial derivatives d k+1 4> / de d k s exist and are continuous functions. 
In particular, it follows from the classical Schwarz's theorem that d 2 4>/deds = d 2 4>/dsde 
and d 3 (j)/deds 2 — d 3 (f)/dsdeds. Now, the classical proofs in [TTJ p. 68 and p. Ill] can be 
traced to prove that [7e,T4] = and that V £ satisfies the Jacobi equation. Finally, to get 
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((HJ from (J23 it suffices to use ([23) to obtain R(w,v)w = -3(v,J(w))j(w) + \w\ 2 (v,T)T 
provided w is a horizontal vector. □ 

2.5. Geometry of surfaces in H . Unless explicitly stated we shall consider surfaces with 
empty boundary. Let E be a C 1 surface immersed in H 1 . The singular set Eo consists of 
those points p € E for which the tangent plane T p E coincides with As Eo is closed and 
has empty interior in E, the regular set E — Eo of E is open and dense in E. It was proved 
in [ini Lem. 1], see also PJ Thm. 1.2], that, for a C 2 surface, the Hausdorff dimension of Eo 
with respect to the Riemannian distance on H 1 is less than or equal to one. In particular, 
the Riemannian area of Eo vanishes. If N is a unit normal vector to E in (H 1 ,^), then we 
can describe the singular set as E = {p € E;iV/j(p) = 0}, where Nh = N — (NfT^T. In 
the regular part E — Eo, we can define the horizontal Gauss map Vh and the characteristic 
vector field Z, by 

(2.9) v h := ^- Z= J(v h ). 

As Z is horizontal and orthogonal to Vh, we conclude that Z is tangent to E. Hence Z p 
generates TpT.Ci'Hp. The integral curves of Z in E — Eo will be called (oriented) characteristic 
curves of E. They are both tangent to E and horizontal. If we define 

(2.10) S:=(N,T)p h -\N h \T, 

then {Z p , S p } is an orthonormal basis of T p E whenever p 6 E — Eo- Moreover, for any 
p G E — Eo we have the orthonormal basis of TpH 1 given by {Z p , (vh)p, T p }. From here we 
deduce the following identities on E — E 

(2.11) \N h \ 2 + (N,T) 2 = 1, K) T = (N,T) S, T T = -\N h \S, 
where U T stands for the projection of a vector field U onto the tangent plane to E. 

Given a C 1 immersed surface E with a unit normal vector N, we define the area of E by 



(2.12) A(E) := / |JV fc 



dE, 



where <iE is the Riemannian area element on E. If E is a C 2 surface bounding a set f2, then 
A(E) coincides with all the notions of perimeter of f2 and area of E introduced by other 
authors, see [H Prop. 2.14], [13 Thm. 5.1] and [H Cor. 7.7]. 

Finally, for a C 2 immersed surface E with a unit normal vector TV, we denote by B the 
Riemannian shape operator of E with respect to N. It is defined for any vector W tangent 
to E by B(W) = —DyyN. The Riemannian mean curvature of E is —2Hr = divs N, where 
divs denotes the Riemannian divergence relative to E. 

2.6. Isometries and dilations. By a horizontal isometry of H 1 we mean an isometry of 
(H , g) leaving invariant the horizontal distribution. These isometries preserve the area de- 
fined in (I2.12[) . Examples of such isometries are the left translations and the Euclidean 
rotations about the t-axis. We say that two surfaces Ei and E2 are congruent if there is a 
horizontal isometry <j) such that 0(Ei) = E2. 

In the Heisenberg group H 1 there is a one-parameter group of C°° dilations {<5a}a£R given 
in coordinates (x, y, t) by 

(2.13) 6 x {x,y,t) = (e x x,e x y,e 2X t). 

From (I2.13P it is easy to check that any S\ preserves the horizontal and the vertical distri- 
butions. The behaviour of the area with respect to 5\ is contained in the formula 

(2.14) A(5 X (Z)) = e 3A A(E). 
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For a proof of (j2~T4"| see [33l Proof of Thm. 4.12]. 

2.7. A weak Riemannian divergence theorem. Let E be a C 2 Riemannian surface. For 
any integer r ^ 1 we denote by Cg(E) and C r (E) the spaces of functions of class C r with or 
without compact support in E. For r ^ 1 let L r (E) be the corresponding space of integrable 
functions with respect to the Riemannian measure dE. Let U be a C 1 tangent vector field 
on E. Given a continuous function / on E, a continuous vector field V on E, and a point 
p G E, we define £7 P (/) = (/ ° a)'(0) and (.D(/V)(p) = V^, s s(0). Here a is the integral curve 
of U with a(0) = p, while the primes denote derivatives of functions depending on s and 
covariant derivatives along a(s). We say that / and V are C 1 in the [/-direction if U(f) 
and DjjV are well defined and they are continuous on E. We also set 



where divs U stands for the Riemannian divergence of U . Note that these definitions coin- 
cide with the classical ones when / G C' 1 (E) and V is a C 1 vector field on E. In the same 
way we can introduce derivatives of higher order in the [/-direction. 

Now we extend the classical Riemannian divergence theorem in E to certain vector fields 
with compact support which are not C 1 on E. First we need an approximation result. 

Lemma 2.3. Let E be a C 2 Riemannian surface. Consider a C 1 tangent vector field U on 
E such that U p ^ for any p£S. Then, for any function f G Cq(E) which is also C 1 in 
the U -direction, there is a compact set K C E and a sequence of functions {f £ }e>o in Co(E) 
such that the supports of f and f £ are contained in K for any e > 0, and 

(i) {f e } —tfin L r (E) for any integer r ^ 1, 

(ii) {U(f £ )} — > U(f) in Z/(E) for any integer r 1. 

Proof. Let p G E. By using the local flow of U in E and that U p ^ 0, we can find a local C 1 
chart (D, <fi = (x, y)) of E around p such that K = D is compact and the restriction of U to 
D coincides with the basic vector field d y . This means that U(h) = (d(h o (f>~ 1 )/dy) o (f> for 
any function h which is C 1 in the [/-direction. To finish the proof it suffices, by a standard 
partition of unity argument, to prove the claim when the support of / is contained in D. Let 
D' = 4>{D) and g = f o We have g £ C (D') and dg/dy = [/(/) o e C (£>')< From 
the standard regularization by convolution in K 2 , see for instance [151 Sect. 4.2.1], we can 
find a sequence {g e }e>o m C^°(K 2 ) such that — > g and {dg E /dy} — > dg/dy uniformly 
in R 2 , while the supports of g e are contained in D' for any e > 0. It follows that the family 
{/e}e>o with f e — g e o satisfies {/ £ } — > / and {[/(/ e )} — > U(f) uniformly in D, while the 
support of f e is contained inDcX for any e > 0. Clearly {/ £ } £ >o proves the lemma. □ 

Lemma 2.4. Le£ E 6e a C 2 Riemannian surface. Consider a C l tangent vector field U 
on E such that U p ^ /or any p G E. Then, for any f G Cq(E) which is also C 1 in the 
U -direction, we have 



Proof. By definition (|2.15[) it follows that divs(fU) G i 1 (S) since / has compact support 
and [/(/) is continuous. By Lemma 12.31 we can find a sequence {/ £ } £ >o in Co(E) such that 
—s- / and {U(f £ )} —> U(f) in L 1 (E), while the supports of f e and / are contained in the 
same compact set K C E for any e > 0. In particular, we deduce {/ e divs U} — > /divs [/ 
in i 1 (E) since divs [/ is continuous. By using the Riemannian divergence theorem for C 1 
vector fields with compact support, we obtain 



(2.15) 



div s (/[/) :=/div E U + U(f), 





e > 0. 
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Letting e — > in the previous equality the claim is proven. 



□ 



3. Stable surfaces. Second variation formulas of the area 

In this section we define stable surfaces and we show that they satisfy an analytical in- 
equality by means of a second variation formula for the area functional defined in (|2.12p . 
We first introduce the appropriate variational background. 

Let E be a C 2 oriented surface immersed in H 1 with singular set Eo. By a variation of 
E we mean a C 1 map ip : I x E — > H , where / is an open interval containing the origin, 
satisfying the following properties: 

(i) ip(0,p) — p for any p 6 E, 

(ii) The set E s = {(p(s,p); p £ E} is a C 1 surface immersed in H 1 for any s£l, 

(iii) The map tp s : E — > E s given by <p s {p) — <p(stP) is a diffeomorphism for any s£/. 

We say that the variation is compactly supported if there is a compact set K C E such that 
ipsip) = P for any s € I and p 6 E — K. If, in addition, the set K is contained inside 
E — Eo then the variation is nonsingular. The area functional associated to the variation is 
A(s) :— A(Y, S ). Note that only the deformation over the compact set K contributes to the 
change of area. We say that E is area- stationary if A'(0) = for any compactly supported 
variation. We say that E is stable (resp. stable under non-singular variations) if it is area- 
stationary and A"(Q) ^ for any compactly supported (resp. non-singular) variation of E. 
Finally by an area-minimizing surface in H 1 we mean a C 2 orientable surface E such that 
any compact region M C E satisfies A(M) ^ A(M') for any other C 1 compact surface M' 
in H 1 with dM — dM' . Clearly any area-minimizing surface is stable. 

Remark 3.1. Consider a C 1 vector field U with compact support on E. For any selwe 
denote <p s (p) = exp p (sJ7 p ), where exp p is the exponential map of (H , g) at p. It is easy to 
see that, for s small enough, {(p s } s defines a compactly supported variation of E. In case 
the support of U is contained in E — Eo then the induced variation is nonsingular. This was 
the point of view used in |33] to define variations of a C 2 surface. In particular, our notion 
of area-stationary surface implies the one introduced in [331 Sect. 4]. 

It is clear that stability is preserved under left translations and vertical rotations since 
they are horizontal isometries in H . In the next result we prove that any dilation S\ as 
defined in ()2 . 13[) satisfies the same property. 

Lemma 3.2. Let E be a C 2 immersed oriented surface in H . Then E is stable (resp. stable 
under non-singular variations) if and only if the same holds for ^a(E). 

Proof. Let E^ = <5a(E). Take a compactly supported variation {ip s } se i of E>. By using 
that the family of dilations is a one-parameter group of diffeomorphisms we can see that 
{tp s } S £i with ip s = 5-\ o tp s o 5\ provides a compactly supported variation of E. Moreover, 
the variation {tp s } s& i is nonsingular if and only if {ip s } se j is nonsingular. By (12.14[) we get 



A(E S )=AW> S (E))=A(5_ A ((E A ) S )) 



A((E A ) S ). 



From here it is easy to deduce that if E is stable (resp. stable under non-singular variations) 
then the same holds for E A . To prove the reverse statement it suffices to change the roles 
of E and E A . □ 
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3.1. Area-stationary surfaces. In this part of the section we gather some facts about 
area-stationary surfaces in H 1 that will be useful in the sequel. 

Let £ be a C 2 immersed surface in H 1 with a unit normal vector N. We define the mean 
curvature of £ as in |32] and |33] , by the equality 

(3.1) -2ff(p) = (div E i/fc)(p), pe£-£ , 

where Vh is the horizontal Gauss map defined in (|2.9j) and divs U stands for the divergence 
relative to £ of a C 1 vector field U. We say that £ is a minimal surface if the mean curvature 
vanishes on £ — £q. 

In the following proposition we recall some features about area-stationary and minimal 
surfaces in H 1 involving the structure of the regular and the singular set, see [TU1 Sect. 3], 
|331 Sect. 4] and the references therein. Similar results also hold in other sub-Riemannian 
spaces, see [26] and [28] . 

Proposition 3.3. Let £ be a C 2 immersed oriented minimal surface in H 1 with singular 
set So- Then we have 

(i) Any characteristic curve of ' £ is a segment of a horizontal straight line. 

(ii) So consists of isolated points and C 1 curves with non-vanishing tangent vector 
{singular curves). 

(iii) If r is a singular curve and p G r, then there is a neighborhood B of p in £ such 
that B — r is the union of two disjoint domains B + and B~ contained in £ — So- 
Moreover, the vector fields Z and extend continuously to p from B + and B~ in 
such a way that Z+ = — Z~ and (vh)p = —{ v h)p- 

(iv) If £ is any C 2 immersed oriented surface, then £ is area- stationary if and only if 
£ is minimal and the characteristic curves meet orthogonally the singular curves. 

Now we prove a regularity result for minimal surfaces in H 1 . Given a C 2 surface £ in H 1 
with unit normal vector N, it is clear that the vector field DzN is well defined on £ — £o 
and it is continuous. By using the ruling property of minimal surfaces in Proposition ^. 31 (i) 
we can obtain more regularity for N in the Z-direction. 

Lemma 3.4. Let £ be a C 2 immersed oriented surface in H . If £ is minimal then, in 
£ — £o, the normal vector N is C°° in the direction of the characteristic field Z . 

Proof. Take p€ £ — £q. Let 7 be the characteristic curve through p. Consider a C 1 curve 
a : (— eo,£o) — > S — £0 transverse to 7 with a(0) = p. Define F{e,s) := a(e) + s Z a t e y By 
using (|2.6|) and Lemma [2.11 we get that V(s) :— (dF/de)(0,s) is a C°° Jacobi field along 
7. Since both 'j(s) and V(s) are C°° and linearly independent for s small enough, the unit 
normal N to £ along 7 is given by 

7 x V 

N = ±yL 

7 x V\ 

where x denotes the cross product in (HI 1 , _g) . We conclude that N is a C°° vector field 
along 7. □ 



3.2. Second variation of the area. In this part of the section we provide some formulas 
for the second derivative of the area functional associated to some variations of an area- 
stationary surface. We first give some preliminary computations. 

Lemma 3.5. Let £ C H 1 be a C 2 immersed surface with unit normal vector N and singular 
set £q- Consider a point p € £ — £q, the horizontal Gauss map Uh and the characteristic 
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field Z defined in (12.9[) . For any v £ TpH 1 we have 



(3.2) D v N h = (D v N) h - (TV, T) J(v) - (N, J(v)) T, 

(3.3) v(\N h \) = (D v N,v h )-(N,T)(j(v),v h ), 

(3.4) v((N,T)) = (D V N,T) + (N,J(v)), 

(3.5) D v v h = \Nh\- 1 ((D V N, Z) - (N, T) (J(v), Z)) Z + (Z, v) T. 



Proof. Equalities ([3721 and (153)) are easily obtained since N h — N - (N, T) T. The proof 
of (|3.4|) is immediate. Let us show that (|3.5[) holds. As \vh\ — 1 and {Z p , (vh) p ,T p } is an 
orthonormal basis of TpH 1 , we get 

D v i/ h = (D v v h , Z) Z + (D v v h , T) T. 

Note that (D v v h ,T) = -(u h ,J(v)) = (Z,v) by {272]). On the other hand, by using ([3~2|l 
and the fact that Z is tangent and horizontal, we deduce 

(D v v h ,Z) = INh^DvN^Z) = INh^^DvN, Z) - (N, T)( J(y), Z)), 

and the proof follows. □ 

Remark 3.6. In E — So we can consider the orthonormal basis {Z, S} defined in (|2.9[) and 
(|2.10p . By using the definition of mean curvature in (|3.ip we have 

-2H = div s v h - {D z v h , Z) + {D s v h , S). 

By ([575]) we get D z v h = T - \N h \~ l (B{Z), Z)Z, and that D s Uh is proportional to Z. It 
follows that, in E — So 

(3.6) 2H=\N h \- 1 (B(Z),Z), 

(3.7) D zVh =T- (2H) Z, 

where B is the Riemannian shape operator of S. On the other hand, the vector D Z Z is 
orthogonal to Z and T since \Z\ = 1 and (J(Z), Z) = 0. It follows that D Z Z is proportional 
to Vh. From p. 7p we obtain 

(3.8) D Z Z - {D Z Z, v h = 2Hv h . 

The second derivative of the area for non-singular variations of a minimal surface in H 1 
has appeared in several contexts, see [TUJ Prop. 6.1], [H Sect. 3.2], (T2] Sect. 14], [301 Proof 
of Thm. 3.5] and Thm. E]. In the next theorem we compute the second derivative of the 
area functional for some non-singular variations by Riemannian geodesies of a C 2 minimal 
surface (maybe with non-empty boundary) in Mr. 

Theorem 3.7. Let E C H 1 be a C 2 immersed minimal surface with boundary <9E and sin- 
gular set Eo- Consider the C l vector field U = vN + wT , where N is a unit normal vector 
to E and v, w € Co(E — Eo). If u = (U, N), then the second derivative of the area for the 
variation induced by U is given by 

(3.9) A"(0) = [ \Nh\- 1 {Z{uf - (\B(Z) + S\ 2 - 4|7V,| 2 ) u 2 } dE 

+ / div s (£Z)dE+ / div s (^Z)rfE. 

Here {Z, S} is the orthonormal basis in (|2.9p and (|2.10p . B is the Riemannian shape oper- 
ator of E, the functions £ and (i are defined by 

(3.10) t = (N,T)(l-(B(Z),S))u 2 , 

(3.11) n = \N h \ 2 «JV, T) (1 - (B(Z),S)) w 2 - 2(B(Z), S) vw) , 
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and the divergence terms are understood in the sense of (|2.15[) . 
In particular, if <9E is empty, then 

(3.12) A"(0) = [ \Nh\- 1 {Z{uf - (\B(Z) + S\ 2 - A\N h \ 2 ) u 2 } dE. 

Proof. We will follow closely the arguments in [37l §9]. Let ip s {p) — exp p (sJ7 p ), for s small, 
be the variation induced by U. Then any E s = y s (E) is a C 1 immersed oriented surface. 
We extend the vector U along the variation by setting U(ip s (p)) — (d/dt)\t= s ftip)- Let N 
be a continuous vector field along the variation whose restriction to any E s is a unit normal 
vector. By using (I2.12j) . the coarea formula, and that the Riemannian area of Eo vanishes, 
we have 

(3.13) A(s)=A(X s )= [ \N h \dX s = [ (|iVft|o^ s )|J a c^ s |dE, 

where Jac</? s is the Jacobian determinant of the diffcomorphism (p s : E — > E s . 

We can suppose that \Nh\(<p„(p)) > whenever p 6 E — E and \s\ < s . Take a point 
p G E — Eo and consider the orthonormal basis {ei,e2} of T p E given by e\ — Z p and 
c-i = S p . Let 7 be the Riemannian geodesic defined by 'y(s) = <p s (p) = cxp p (sf/ p ). De- 
note by N(s) the unit normal to E s at 7(s). Let on : (— £oj£o) — > E — Eo be a C 1 curve 
such that ai(0) = p and dj(0) = ej. We define the C 1 map F{ : {— £o,£o) x M — > H 1 
given by Fi(e,s) — ip s (ai(e)) — exp ai f e \(sU ai ^). By using Lemma [2.11 we deduce that 
Ei(s) = (dFi/de)(0, s) = ei(<p s ) is a C°° Jacobi vector field along 7 with [7, = and 
Ei(0) = ei. Therefore, we have the following identities along 7 

(3.14) D u D u E i = -R(U,E i )U, 

(3.15) D u E i = D Ei U. 

On the other hand, it is clear that {Ei(s), E2(s)} provide a basis of the tangent space to 
E s at 7(s). In particular |Jaccp s | = (|£:i| 2 \E 2 \ 2 - (E 1: E 2 ) 2 ) 1/2 {s), and so |Jac</3 s | is C°° 
along 7. Moreover, we have N(s) = ±\Ei x E 2 \~ x {E\ x i? 2 )(s), which is C°° on 7. Here 
x is the cross product in (EI 1 ,^). We conclude that |JV/,|(a) is C°° along 7 as well. Thus 
we can apply the classical result of differentiation under the integral sign to deduce, from 
(|3~T3"|) . that 

(3.16) A"(0)= f {|AT /l |"(0) + 2|Ar h | / (0)|Jac^|'(0) + |Ar fc ||Jac^r(0)}dE ! 

where we have used that fo{p) = P for any p £ E, and so IJac^ol = L 

Now we compute the different terms in (|3.16[) . The calculus of | Jacy s |'(0) and |Jac<^ s |"(0) 
is found in [37l §9] for C 2 variations of a C 1 surface in Euclidean space. The arguments can 
be generalized to any Riemannian manifold for a C 1 variation obtained when we leave from 
a C 2 surface by geodesies. As U = vN + wT on E, we deduce, by using divs T = and the 
third equality in (|2.1ip . that 

(3.17) |Jacp.|'(0) = div E U - (-2H R ) v - \N h \ S(w) = -(B(S), S) v - \N h \ S(w). 
To get the second equality we have taken into account (|3.6p to obtain 



2H R = - div E N = (B(Z),Z) + (B(S), S) = (B(S), S). 
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On the other hand, it is known that 

2 

|Jacv? s |"(0)=(div s U) 2 + J2\( D e i U) X \ 2 

i=l 

2 2 

-J2(RiU,ei)U,ei)- ^ (D et U,e j )(D ej U,e i ). 

i—l — \ 

Hence from (13.17[) . equality 

(3.18) D e U = e(v) N-v B{e) + e(w) T + w J(e), 
and equation (|2.3[) . we get 

(3.19) |Jac^|"(0) =|V E «| 2 + (N,T) 2 |V s w| 2 - 2\N h \ Z(y) w - 2\N h \ (B(Z), S) Z(w) v 

+ 2(N, T) Z{v) Z{w) + 2(N, T) S(v) S(w) 

- (Ric(iV, N) + \B\ 2 - (B(S),S) 2 ) v 2 - 4(N, T) vw, 

where Vs is the gradient relative to E, Ric is the Ricci tensor in (H 1 ,^), and \B\ 2 is the 
squared norm of the Riemannian shape operator of E. 

Let us compute \N h \'(0) and \N h \"(0). From (|3~3)l and (f2T2l) it follows that 

\N h \'(a) = U(\N h \) = {D V N, v h ) - (N, T) (J(U),v h ) = (D V N, v h ) + (N, T) (U, Z). 

Note that U — uN — {\N h \w)S. Then (U,Z) = and D V N = -V s u + (\N h \w)B(S) on 
E — Eo. By the second equality in (|2 . 1 1 1) we obtain 

(3.20) \N h \'(0) = (D LJ N, Vh ) = -(N,T)S{u) + \N h \(N,T)(B{S),S)w. 
We also deduce the following 

(3.21) \N h \"{Q) =(DuDuN, v h ) + (D V N, D v v h ) 

+ U((N, T» (U, Z) + (N, T) U((U, Z)) 
=(DuDuN, v h ) + (DuN, D v v h ) + (N, T) (U, D V Z), 

since (U,Z} = and DjjU — on E — Eq. We can compute DjjVh from (|3.5[) . By using 
that DyiV = -V s u + (|iV h |io)5(5) and J([T) = {\N h \v)Z on E - E , we get 

D uVh = -l^r 1 (Z(t») - \N h \ (B(Z), S) w + \N h \ (N, T) v) Z, 

and so 

(3.22) (D V N, D uVh ) H^r 1 Z{u) 2 + (N, T) Z(u) v - 2(B(Z), S) Z(u) w 

- |JV A | (N, T) (B(Z), S) vw + \N h \ (B{Z),S) 2 w 2 . 

Now we compute DjjZ. The coordinates of this vector with respect to the orthonormal basis 
{Z,v h ,T} are given by 

(D V Z,Z) = Q, (DuZ, Vh ) = \Nh\- 1 Z(u)-(B(Z),S)w+(N,T)v, (D V Z,T) = -\N h \v. 
The previous equalities and the fact that U = vN + wT on E — Eo imply that 

(3.23) (U,DuZ) = Z(u)v-\N h \(l + (B(Z),S))vw. 

It remains to compute DjjDjjN. Note that {Ei, E2, N} provides an orthonormal basis of 
TpM 1 . As a consequence 

2 

DuDuN = (DuDuN, E t ) E, + (DuDuN, N) N. 
i=l 
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As (iV, Ei) = along 7 we get 

(DuDuN, Ei) = -2(DuN,DuEi) - (N,DuDvEi) 
= -2(D u N,D Ei U) + (N,R(U,E i )U). 

The second equality follows from p,15[) and (I3.14p . Now recall that e\ = Z p and e2 = S p . 
It follows that 

(3.24) {DuDuN, v h ) = - 2(N, T) (D v N, D s U) + (N, T) (N, R{U, S)U) 

+ \N h \{D u D u N 1 N). 

By taking into account (|3.18[) and that DjjN = — V^u + (\Nh\ w)B(S), we obtain 

(3.25) (DvN, D S U) = - \N h \ 2 (B(S),S) S{w) w - (N, T) Z{u) w 

+ \N h \S{u)S{w) + B(S)(u)v 
+ \N h \(N,T) (B(Z),S)tv 2 -\N h \\B(S)\ 2 vw. 
On the other hand, we use (|2.3p so that, after a straightforward computation, we conclude 

(3.26) (R(U, S)U, N) = \N h \ {v + (N, T) w) w. 
Moreover, since |JV| 2 = 1 on E — So we have 

(3.27) (DuDuN, N) = -\D V N\ 2 = -|V E u| 2 + 2\N h \ B{S){u) w - \N h \ 2 \B{S)\ 2 w 2 . 

By substituting (j3~23|) . (j3~2^| and (j3~271) into (j3~2^| we get (DvDvN,v h ). From (j3~Ml) . 
(13.22)1 and (|3.23|) . after simplifying, equality (I3.2ip becomes 



(3.28) \N h \"(0) ^N^ 1 Z(u) 2 - \N h \ |V s u| 2 + 2<iV, T) (Z(u) v - B(S)(u) v) 

+ 2((N, T) 2 + liV^I 2 (B(Z), S) - (B(Z),S)) Z(u) w 

+ 2\N h \ 2 (B(S), S) (S(u) w + (A, T) S{w) w) - 2\N h \ (A, T> S(u) S(w) 

+ 2\N h \ (N,T) (\B(S)\ 2 - (B(Z), S)) vw 

+ (\N h \ (A, T) 2 (1 - (B(Z), S)) 2 - \N h \ 3 (B(S),S) 2 ) w 2 . 

Now, since u = v + (N,T)w, we have V s u = VsU + iu V S ((A,T)) + (N,T)V s w. By 
(|3.4p and (|2.1ip it is easy to see that 

(3.29) Z((N, T)) = \N h \ ((B(Z), S) - 1), 

S((N,T)) = \N h \(B(S),S). 

This allows us to compute the term \Nh\ |Veu| 2 in (|3.28[) . At this moment, we use f|3 . 28[) . 
(|3.20p . (|3.17p and (|3.19p so that, after simplifying, we get that 

\N h \"(0) + 2 |JV h |'(0) |Jac^|'(0) + \N h \ |Jac^|"(0) 

is equal to 

(3.30) \Nh\~ 1 Z{uf + 2(N, T) (1 - (B(Z),S)) (Z(v) v + Z{w) w) 

+ 2((N, T) 2 - (B{Z),S)) (Z(w) v + Z(v) w) + q x v 2 
+ 2\N h \ (N, T) ((B(Z), S) -3)vw- \N h \(l - (B(Z), S)) 2 w 2 , 
where q± is the function given by 

qi = \N h \ ((B(S),S) 2 -Ric(A, A) - \B\ 2 ). 
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In order to obtain (I3.9|) from f|3.16[) and (|3.30[) . we apply Lemma f3T9l below. We deduce 
the following 

|JV h |"(0) + 2 |JV h |'(0) |Jac^|'(0) + |JVh| |Jac^ s |"(0) 
= |iV, l |- 1 Z( M ) 2 + div s (pZ) 

+ { Ql + ((B(Z),S) - 1) ((iV, T) q 2 + Z((N, T») + (iV, T) 5))} V 2 

+ { {(B(Z),S) - 1) ((iV, T) q 2 + Z((N, T») + (JV, T> 5)) 
-|JVfc| (1 - <B(Z), S)) 2 } w 2 + 2 {|JV h | (iV,T) ((B(Z),S) - 3) - Z((iV,T) 2 ) 

-(AT, T) 2 g 2 + S) q 2 + Z((B(Z),S))}vw, 

where p is the function 

(N, T) (1 - (B(Z), S)) (v 2 +w 2 )+2 ((N, T) 2 - (B(Z),S))vw. 
A straightforward computation using (|3.29[) . (|3.34p . the identities 
Ric(7V, N) = 2 — 4\N h \ 2 (it follows from flU}), 

\B\ 2 = (B{Z), Z) 2 + (B(S), S) 2 + 2(B(Z), S) 2 = (B(S), S) 2 + 2(B(Z), S) 2 , 
B(Z) = (B(Z), Z)Z + (B(Z), S)S= (B(Z), S) S, 
and that u = v + (N, T^w, gives us 

|JV h |"(0) + 2\N h \'(Q) |Jac<^|'(0) + \N h \ |Jac^|"(0) 

=\N h \~ l Z{u) 2 - \N h \~ l (\B(Z) + S\ 2 - A\N h \ 2 ) u 2 
+ div s (e^)+div s (^Z), 
where £ and /i are the functions given in (|3.10l) and (13. lip . 

Finally, suppose that <9£ is empty. Then £ and /x are continuous functions with compact 
support in E — So and they are also C 1 in the Z-direction by Lemma T3. 101 Hence the inte- 
grals of divs(^Z) and divs(/iZ) vanish by virtue of the divergence theorem in Lemma |2. 41 
This proves ((3321 . □ 

Remark 3.8. The divergence terms in (|3.9p need not vanish if <9£ is nonempty. In the 
proof of Proposition 15.21 we will show that these terms play an important role. 

Lemma 3.9. Let £ be a C 2 immersed oriented surface in H 1 and <f> G C (£). Then, in the 
regular set £ — T,q, we have 

div s (0Z) = Z{(j>) + q 2 cf>, 

where q 2 is the function given by 

q 2 = \N h \- 1 (N,T)(l + (B(Z) 1 S)). 

Proof. Clearly we have 

(3.31) div E (0Z) = (div s Z)0 + Z(0). 

Note that 

div s Z = (D Z Z, Z) + (D S Z, S) = (D S Z, S), 

since \Z\ 2 — 1. We compute the components of D$Z in the orthonormal basis {Z,Vh,T}. 
Observe that DgZ is orthogonal to Z. By using (13.51) and that J(S) = (N,TjZ : we get 

{D s Z,v h ) = ~{Z,D s v h ) = \Nh\~ 1 ((B(Z),S) + (^T) 2 ), 
(D S Z,T) = -(Z,J(S)) = -(N,T). 
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From here we deduce 

(3.32) D S Z = \Nh\- 1 {(B{Z),S) + 1 - |^| 2 ) v h - (N,T) T. 
As a consequence, we obtain 

(3.33) div E Z = (D S Z,S) = \N h \- 1 (N,T)(l + (B(Z),S)). 

The proof finishes by substituting f|3 . 33[) into (|3.31|) . □ 

Lemma 3.10. Let E be a C 2 immersed oriented minimal surface in H . Then, in the 
regular set E — Eo, we have 

(i) The functions (N,T\ and \Nh\ are C°° in the Z -direction. 

(ii) The vector fields and S are C°° in the Z -direction. 

(iii) The function (B(Z),S) is C°° in the Z-direction, and 

(3.34) Z((B(Z), S)) = \\N h \ (N,T) - 2\N h \~ 1 (N, T) (B(Z),S) (1 + (B{Z),S)). 

Proof. Recall that ./V is C°° in the ^-direction by Lemma T3. 41 This implies (i). Assertions 
(ii) and (iii) follow from (i) by the definition of Vh and S in (I2.9[) and (|2.10p . To compute 
Z((B(Z),S)) note that 

Z((B(Z), S)) = Z(-(D Z N, S)) = -(D Z D Z N, S) - (D Z N, D Z S). 

It is clear that D Z N is tangent to E. On the other hand, D Z S is proportional to N. This 
comes from the fact that (D Z S,Z) = -(S,D Z Z) = by whereas (D Z S,S) = 0. 

Therefore we have 

(3.35) (D z N,D z S) = 0, 

(3.36) Z{(B{Z),S)) = -{D Z D Z N, S) = (N, D Z D Z S). 

It remains to compute D Z D Z S. From (|3 . 32[) we see that D$Z is C°° in the Z-direction. 
As a consequence [Z,S] = D Z S — T)gZ is also C°° in the Z direction, and D Z [Z,S] = 
D Z D Z S — D z DsZ . Thus equation (|3.36p becomes 

(3.37) Z((B(Z), S)) = (A, D Z [Z, S]) + (N, D z D s Z) 

= (A, D Z [Z, S]) + (A, D S D Z Z) - (A, R(Z, S)Z) + (A, D [ZS] Z) 

= (A, D Z [Z, S}) - (A, R(Z, S)Z) + (A, D [Z ^Z), 

where R is the Riemannian curvature tensor and we have used (|3.8[) to get D$D Z Z = 0. 
Now, observe that 

([Z, S],N) = (D Z S, A) - (D 8 Z, A) = -{S, D Z N) + (Z, D S N) = 0, 

which implies that [Z, S] is tangent to E. Therefore, we deduce 

(A, D Z [Z, S}) = (B(Z), [Z, S}) = (B(Z), D Z S) - (B(Z),D S Z) = -(B(Z), D S Z), 

(N,D [ZS] Z) = -(D [ZS] N,Z) = (B(Z),[Z, S}) = -(B(Z),D S Z), 

where we have used (|3.35[) . If we put this information into (I3.37[) , we obtain 

(3.38) Z((B(Z),S}) - -2(B{Z),D S Z) - (A, R{Z, S)Z). 

To compute the first term above we take into account (|3.32p . After simplifying, we get 

(3.39) (B(Z), D S Z) = \Nh\~ 1 (A, T) (B(Z), S) (1 + (B(Z), S)). 

For the second term, we apply (|2.3p so that, after a straightforward calculus, we conclude 

(3.40) (N,R(Z,S)Z) = -4\N h \(N,T). 

The proof finishes by substituting (PPS]) and ([3~4Hj) into (|3~3^|) . □ 
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In the next result we compute the second derivative of the area for some vertical varia- 
tions of an area-stationary surface E whose singular cuves (Eq) c are C 3 (in [33l Prop. 4.20] 
we proved that they are always C 2 ). We suppose that the variation is constant along the 
characteristic curves of a tubular neighborhood around (£o)c- By a tubular neighborhood 
of radius e > we mean the union of all the characteristic segments of length 2e centered 
at (S )c- 

Proposition 3.11. Let E be a C 2 immersed oriented area- stationary surface in H 1 such 
that the singular curves (Eo) c of E are of class C 3 . Let <p r (p) '■— ^Pp( r w(p)Tp), for r 
small, be the vertical variation of E induced by a function w £ Cq(E). Suppose that there 
is a tubular neighborhood Eq o/supp(w) H (So) c where Z{w) = 0. Then, there is a tubular 
neighborhood E o/supp(w) fl (£o) c such that 

A(<p r (E))= [ S(w) 2 dl, 

where S is any continuous extension of the vector field S defined in (|2.10p to (£o) c and dl 
denotes the Riemannian length element. 

Proof. We can restrict ourselves to a neighborhood of a single singular curve T. We con- 
sider a parameterization T(e) = (x(e),y(e),t(e)) by arc-length. By Proposition 13.31 the 
area-stationary surface E can be parameterized in a neighborhood of supp(w) (~l T by 

(e,s)->r(e) + «J(f(e)), 

so that the curves with e constant are the characteristic curves of E. In Euclidean coordi- 
nates we have 

x(e,s) = x(e) - sy(e), 

Vfas) = y( £ ) + s %( £ ), 

t(e,s) = t(e)-s(xx + yy) (e). 

As Z(w) = we get that w is a function of s alone. The deformation <p r (p) — e:x P P { rw {p)Tp) 
consists on changing the i-coordinate of the above parameterization by 

t(e, s) + rw(e). 

A simple computation shows that the tangent space to the surface E r := yv(E) is generated 
by the vectors 

(3.41) -yX + xY, (x~sy)X + {y + sx)Y+(s(-2 + sh)+rw)T, 

where x, y and t are the coordinates of T, dots represent derivatives with respect to e, and 
h = h{e) = (xy — yx){e) is the Euclidean geodesic curvature of the a;y-projection of P. 

Hence the singular points of E r corresponds to the zero set of F(e, s, r) := s(— 2 + sh(e)) + 
rw(e). Observe that F is a C 1 function since the singular curves are assumed to be of class 
C 3 and w € C 2 . As (dF/ds)(e, 0, 0) = —2, we can apply the Implicit Function Theorem and 
a compactness argument to show that there are positive values So, So, ro, and a C 1 function 
s : (— £o,£o) x (—r ,r ) —> (—so, so) with s(e, 0) = satisfying F(e, s(e,r),r) — 0. Here 
eo > is taken so that supp(w) HTc [— £o,£o]- We define E := F((— £o,£o) x (— so,So))- 

On the other hand, a computation using p. 411) shows that 

\(N h ) r \dY, r = \s(-2 + sh(e))+rw(e)\dsds. 

Hence we have 

A(tp r (E))= f° \ f° \s(-2 + sh{e))+rw(e)\ds\de. 



18 



A. HURTADO, M. RITORE, AND C. ROSALES 



Denote by f £ (r) the integral between brackets. As (dF/ds)(e, 0, 0) < we deduce 

/s(e,r) ps 
(s(-2 + sh(e)) + rw(e)) ds + (s(2 - sh(e)) - rw(e)) ds. 

-s Js(e.r) 

Taking derivatives with respect to r we obtain 

/s(e,r) rso 
w(s)ds— / w(e) ds = 2w(s)s(s,r). 
-so J s(e,r) 

Taking derivatives again we have 

Since (ds/dr)(e,0) = w(e)/2 we conclude 
and so 



dr 2 



so 

2 



A(p r (£0) = / w{eyde. 

r=0 J—eo 

By Proposition 13.31 we know that the vector field S defined in (|2.10l) extends continuously 
to r as a unit tangent vector to T. Then w(e) 2 = S(w) 2 and the claim follows. □ 



3.3. A stability criterion for stable surfaces in H 1 . Here we obtain a useful criterion 
to check if a given area-stationary surface is unstable. First we need a definition. Let £ be 
a C 2 oriented minimal surface immersed in H . For two functions u,v £ Co(£ — £o) which 
are also C 1 in the Z-direction, we denote 

(3.42) I(u, v) := f \Nh\~ 1 {Z{u) Z{v) - (\B(Z) + S\ 2 - 4\N h \ 2 ) uv} tZE, 



s 

where {Z, S} is the orthonormal basis in (12.91) and (12 . 10|) . and B is the Riemannian shape 
operator of S. The expression Q3.42p defines a symmetric bilinear form, which we call the 
index form associated to £ by analogy with the Riemannian situation, see [3] . 

Proposition 3.12. Let £ be a C 2 immersed oriented area- stationary surface in H 1 with 
singular set £o- //£ is stable under non-singular variations then the index form defined in 
(|3.42p satisfies X(tt, u) for any function u 6 Co(£ — £o) which is also C 1 in the direction 
of the characteristic field Z . 

Proof. Let N be the unit normal vector to £. Take u G Cq(£ — £o) and consider the 
vector field U — uN. Note that £ is a minimal surface since it is area-stationary. Hence 
Theorem 13.71 implies that the second derivative of the area for the variation induced by U 
is A"(0) = T(u,u). As £ is stable under non-singular variations we deduce that 

(3.43) X(u,u)^0, for any u £ C£(£ - £ ). 



Now fix a function u £ Co(£— £o) which is also C 1 in the Z-direction. By using Lemma l2~3l 
and that £o has vanishing Riemannian area, we can find a compact set K C £ — £ and a 
sequence of functions {u £ } £> o in Cq(£ — £o) such that {u £ } — > u in L 2 (£), {Z(u £ )} — > Z(u) 
in L 2 (£), while the supports of u £ and u are contained in K for any e > 0. From here 
it is not difficult to check that HN^- 1 / 2 Z{u £ )} -> \Nu\- 1 ! 2 Z{u), {{\N h \^ h) 1 / 2 ^} -> 
[Wh^hY^u and {(|AT h |-i/ 2 )i/2 Ue } -»> (lA,!- 1 ^) 1 / 2 ^ in L 2 (£), where /i = \B(Z) + S\ 2 
and fi = 4|A^/ l | 2 . It follows that lim^o I(u £ , u £ ) = T(u, u), so that inequality p.43p proves 
the claim. □ 

Remark 3.13. As in [T^l Thm. 15.2] and [3D1 Thm. 3.5, Cor. 3.7] the previous result can 
be seen as a Poincare type inequality for stable surfaces in H 1 . 
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3.4. Integration by parts. The stability operator in H 1 . In Riemannian geometry the 
index form of a minimal surface can be expressed in terms of a second order elliptic operator 
defined on the surface, see [3]. In this part of the section we prove a similar property for 
the index form p.42j) of a minimal surface in H 1 which involves a hypoelliptic second order 
differential operator on the surface. 

Proposition 3.14 (Integration by parts I). Let S C H 1 be a C 2 immersed surface with 
unit normal vector N and singular set So. Consider two functions u 6 Co (S — So) and 
v G C(S — So) which are C 1 and C 2 in the Z -direction, respectively. Then we have 

X(u,v) = — / uC(v)dTi, 

where X is the index form defined in (|3.42p . and C is the second order differential operator 

(3.44) C(v) := \N h \- 1 {Z{Z(v))+2\N h \- 1 (N,T)(B(Z),S)Z(v) 

+ (\B(Z) + S\ 2 ~<i\N h \ 2 )v}. 

Proof. Along this proof we shall denote q = \B(Z) + S\ 2 — 4|iV/j| 2 . First note that in 
S — So the hypotheses about u and v ensure that \Nh\~ 1 Z(v) and \Nh\~ 1 Z(v)u are C 1 in 
the Z-direction. Suppose proved that 

(3.45) C(v) = diw(\N h \- 1 Z(v) Z) + |JV h |" V 

In such a case, we would apply the divergence theorem in Lemma 12.41 in order to get 

0= { div s (\N h \- 1 Z(v)uZ)dX= j u Aiv^N^- 1 Z{v) Z) dT, + j \Nt\~ 1 Z{u) Z(v) dS. 

= / uC[v) dS + I{u, v), 

and this would finish the proof. 
To obtain (|3.45p observe that 

(3.46) divjjfliVfcr 1 ^) Z) = iNhl^Ziv) div s Z + Z(\N h \- l Z{v)). 
The computation of divs Z is given in (|3.33p . On the other hand, we have 

(3.47) ZQN^Ziv)) = iN^ZiZiv)) + ZQN^ 1 ) Z(v) 

= \N h \- l Z{Z{v)) - \N h \- 2 Z(\N h \) Z(v) 

= \N h \- l Z{Z[v)) + \N h \- 2 (N, T) ((B(Z),S) - 1) Z(v), 

where we have used (|3.3[) to compute Z(\Nh\). To deduce (|3.45p it suffices to simplify in 
(|3.46[) after substituting the information of (|3.33[) and (|3.47p . □ 

Remark 3.15. If S is a minimal surface then the functional £ in (|3.44p provides a Sturm- 
Liouville differential operator along any of the characteristic segments of S. 

As a direct consequence of Proposition 13. 141 and Proposition 13. 121 we deduce 

Corollary 3.16. Let S be a C 2 immersed oriented area- stationary surface in H . If S is 
stable under non-singular variations then we have 

- J uC{u)dT, 0, 
for any function u G C*o(S — So) which is also C 2 in the Z -direction. 

Finally, with the same technique as in Proposition ^. 14l we can prove the following lemma. 
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Lemma 3.17 (Integration by parts II). Let E C H 1 be a C 2 immersed surface with unit nor- 
mal vector N and singular set So. Consider two functions u € Co(E — So) andv € C(E — So) 
which are C 1 and C 2 in the Z-direction, respectively. Then we have 

\N h \ {Z(u)Z(v) + uZ(Z(v)) + 2\N h \- 1 (N,T)uZ(v)} dS = 0. 

Proof. Observe that in S — So 

div E (|JV h | u Z{v) Z) = u Z(v) {Z(\N h \) + \N h \ div E Z) 

+ \N h \Z(u)Z{v) + \N h \uZ(Z(v)), 

and that the function in the left-hand side has vanishing integral by Lemma 12.41 On the 
other hand, (|3.3I) gives us 

(3.48) Z(\N h \) = (N,T) - (N,T) (B(Z),S), 

which together with (pP3"]) implies Z{\N h \) + \N h \ div s Z = 2 (N, T). The result follows. □ 
Remark 3.18. Some other integration by parts formulas in H 1 can be found in [T21 Sect. 10]. 

4. Complete stable surfaces with empty singular set 

In this section we provide the classification of C 2 complete stable surfaces in H 1 with 
empty singular set. Recall that if So = then S is area-stationary if and only if S is min- 
imal by Proposition 13.31 (iv). We say that an immersed surface S in H 1 is complete if it is 
complete in the Riemannian manifold (H , g). For a C 2 complete area-stationary surface S 
with So = the characteristic curves are straight lines by Proposition 13.31 (i). In particular 
S cannot be compact. Some classification results for area-stationary surfaces with empty 
singular set can be found in [221 Thm. 5.4], [S] and [321 Prop. 6.16]. Note also that for such 
surfaces to be stable is equivalent to be stable under non-singular variations. 

In Euclidean three-space the description of complete stable area-stationary surfaces can 
be obtained by means of a logarithmic cut-off of the function u — 1 associated to the varia- 
tion by level surfaces of the distance function, see [TB] . In H 1 the vector field induced by the 
family of equidistants for the Carnot-Caratheodory distance d cc to a C 2 surface with empty 
singular set coincides, up to a sign, with the horizontal Gauss map i/h, see [TJ Thm. 1.1 and 
1.2]. This leads us to use the stability condition in Proposition 13. 121 with a test function of 
the form / = u|JV/j|, where / is continuous with compact support on the surface and C 1 in 
the direction of the characteristic field Z . We first compute the index form for these type 
of functions. 

Lemma 4.1. Let S C H 1 be a C 2 immersed minimal surface in H 1 with unit normal vector 
N and singular set So- Then, for any function f G Co(S — So) which is also C 1 in the 
Z-direction, we have 

(4.1) X(f\N h \,f\N h \) =J\N h \ {Z(f) 2 - C(\N h \) f 2 } dE, 

where T is the index form in (|3.42j) . and C is the differential operator in (|3.44D . 

Proof. Along this proof we shall denote w = f\N h \ and q = \B{Z) + S\ 2 - i\N h \ 2 . Note 
that w is C 1 in the Z-direction and Z(w) — fZ(\Nh\) + \Nh\Z(f ). If we introduce w in the 
index form we obtain 

(4.2) I(w,w)= [ {\N h \Z(f) 2 + \N h \- 1 Z(\N h \) 2 f 2 + Z{f 2 ) Z(\N h \) - \N h \qf 2 } di:. 
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On the other hand, we know from Lemma 13.101 (i) that \Nh\ is C°° in the Z-direction. 
Therefore, we can apply Proposition 13. 141 with u = f 2 \Nh\ and v = \Nh\, so that we get 



\N h \£(\N h \)fd2 = / {\N h \- l Z(\N h \f f + Z{f)Z(\N h \)-\N h \qf}dV 
= l(w,w) - f \N h \Z{f) 2 d^ 
where in the second equality we have used (14. 2p . This proves the claim. □ 



Remark 4.2. Some other versions of (|4.1j) for variations of a C 2 surface S with associated 
vector field fi/h, f G C 2 (E — S ), can be found in [T31 Lem. 3.9] and [TSl Thm. 3.4]. See also 
[3J Sect. 3.2] and [301 Thm. 3.5] for the case of an intrinsic graph associated to a function 
with less regularity than C 2 . 



In the next lemma we particularize (|4.1[) for / = uv 1 . This type of test functions will 
be used to prove Theorem 14.71 

Lemma 4.3. Let Scl 1 be a C 2 immersed minimal surface in H 1 with unit normal vector 
N and singular set So- Consider two functions ti g (7o(S - So) and v G C(S — So) which 
are C 1 and C 2 in the Z -direction, respectively. If v never vanishes, then 

(4.3) liuv-^Nhluv-^Nhl) = / \N h \v- 2 Z{u) 2 dZ 



- /_ \N h \ uHZiv- 1 ? - \ Z(Z(v- 2 )) - \N h \~ x (N,T) Z(v- 2 ) X dS 

iNhlLdNhDiuv^fdZ, 
where I is the index form in (13.42[) . and C is the differential operator in (|3. 441) . 

Proof. From (|4. 1 1) we only have to compute 

\N h \ Z{uv- x ) 2 dY>. 
Since 

Z{uv- X ) 2 = v- 2 Z{u) 2 + u 2 Z(v- x ) 2 + i Z{u 2 ) Z(v- 2 ), 
and Lemma |3 . 1 71 implies 

i \N h \ Z(u 2 ) Z(v- 2 ) rfS = - £ |JV h | u 2 |i Z(Z{v- 2 )) + liV^- 1 (AT,T) Z(«- 2 )| dS, 

we see that |g3| holds. □ 

The previous lemmas suggest that, for a function u — f\Nh\, the stability condition in 
Proposition l3.12l is more restrictive if £(\Nh\) > 0. Thus it is interesting to compute £(|iV^|) 
and to study its sign. 

Lemma 4.4. Let S be a C 2 immersed minimal surface in H 1 with unit normal vector N . 
Consider the basis {Z, S} defined in (|2.9p and (|2.10p . Let B be the Riemannian shape 
operator o/S. Then, in the regular set S — So, we have 

(4-4) C(\N h \)^4(\N h r 2 (B(Z),S)-l), 

where C is the second order operator in (|3.44p . 



22 



A. HURTADO, M. RITORE, AND C. ROSALES 



Proof. From Lemma fe-jOK i) we know that \Nh\ is C°° in the Z-direction. We must compute 
Z{\N h \) and Z(Z(\N h \)). By (l3~48l) we have 

Z(\N h \) = (N,T)-(N,T)(B(Z),S), 

and so 

Z(Z(\N h \)) = Z((N, T» - Z((N, T» <B(Z), 5) - (iV, T) Z((B(Z), S)). 
Now we use Q3.29P and f|3.34[) , so that we get 

(4.5) Z(Z{\N h \)) = -5\N h \ + A\N h \ 3 + 2|iV /l |- 1 (B(Z), S) 

+ 2|iV h |- 1 (B(Z), S) 2 - 3\N h \ (B(Z), S) 2 . 

By substituting ([3~48)) and (J43J) into ([3~44)) . we obtain 

C(\N h \) = -5 - <B(Z), S) 2 + 4\N h \- 2 (B(Z), S) - 2(B(Z),S)) + \B{Z) + S\ 2 

= 4(|^|- 2 (B(Z),S)-1), 

where in the second equality we have applied that £ is minimal, and so B(Z) = (B{Z\ S^S 
by dSU). This proves $4~4§. □ 



In the next result we show some properties of the Jacobi field associated to the family of 
characteristic segments of a minimal surface in H . This will allows us to study the sign of 
jC ( | TV/i, | ) and to construct suitable test functions to introduce in (|4.3[) when £ is a complete 
minimal surface with empty singular set. 

Lemma 4.5. Let £ C H 1 be a C 2 immersed minimal surface with unit normal N and sin- 
gular set Eo • Consider an integral curve V : I — > £ — So of the vector field S in (|2.10[) . We 
define the map F : I x /' ->• £ - £ by F{e,s) := T(e) + sZ r[e) . Let V e (s) := (dF/de)(e,s). 
Then V £ is a C°° Jacobi vector field along 7 £ (s) :— F(s,s). Moreover, we have 

(i) The vertical component ofV e is given by (V E ,T}(s) = a e s 2 + b E s + c e , with 

b 2 - 4a e c e = -|^| 2 (r(e)) C(\N h \)(T(e)). 

(ii) V e is always orthogonal to "f e and never vanishes along "f e . 

(iii) The function v e {s) := \(V e , T^s)] 1 ' 2 satisfies 

Ziv- 1 ) 2 - \ Z{Z{v- 2 )) \N h \~\N, T) Z{v- 2 ) = M y^ Nhl C(\N h \), 

along any segment 7 e (s) where (V £ ,T}(s) never vanishes. 

Proof. To simplify the notation we will avoid the subscript e along the proof. We will use 
primes for both the derivative of functions depending on s and the covariant derivative along 
7(s). By Proposition [373] (i) the curve 7 is a characteristic curve of £. It follows from (|2.6p 
and Lemma \2. II that V is a C°° Jacobi field along 7 with [7, V] = 0. Note that 

(4.6) (V,T)' = (V',T) + (V,T') = -2(V,v h ), 
since T" = J(Z) = —Vh> an d 

(4.7) (V',T) = (D Z V,T) = {D V Z,T) = -(Z,J(V)) = (J(Z),V) = -{V,v h ). 
If we derive again in (|4.6p then we obtain 

(4.8) (V, T>" = -2{V\ v h ) - 2(V, v' h ) = -2((V, v h ) + (V, T» , 
since v' h = Dz^h — T by (|3.7p and the fact that E is minimal. Hence 

(4.9) (-1/2)(V,T)"' = {{V',v h ) + (V,T))' = (V", v h ) + 2(V,T) + (V,T') = 0, 
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where we have used the Jacobi equation (|2.8I) . equality (|4.7p . and that T' = —Vh- To simplify 
(|4.8I) we compute (V\i>h)- By (|3 . 5|) and the fact that V is tangent to E, we deduce 

(V',vh) = (B v Z,v h ) = -(Z,D v vh) = -\Nh\- 1 {(D V N,Z) - (N,T) (J{V), Z)) 

= \Nh\- 1 {(B(Z),V) + (N,T) (V,v h )), 

and so, after substituting into (|4.8|l . we get 

(4.10) (V,T>" = ^liV^- 1 ((fl(Z),y) + (iV,T) (7,^) + |JV h | (V,T». 

From (|4.9p we conclude that (V,T\(s) is a polynomial of degree at most two. Write 

(4.11) (V,T){s) = as 2 +bs + c. 

Denote p = F(e). As V(0) = S p , it is easy to check by (|4T5]l and gTTU]) . that 

c=(V,T)(0) = -|JV),|(p) 1 

6 = (V,T)'(0) = -2<V,^)(p) = -2(N,T)(p), 

a = (1/2) <V,T)"(0) = -\N h \-\(B{Z),S) + (N,T) 2 - \N h \ 2 )(p). 



In particular, it follows from (|4.4I) that 

6 2 _ 4ac = ^ 4 5 ) _ \N h \ 2 )(p) = -\N h \ 2 (p) C(\N h \)(p), 

which proves assertion (i) in the statement. 
To prove assertion (ii), observe that 

(V, 7>' = (V, 7) + (V, 7') = (D V Z, Z) + (V, D Z Z) = 0, 

by (|3.8[) . This implies that (V, 7) = along 7 since V(0) — S p . Hence there is a C 1 function 
/ : I' -> K such that V = /S along 7. Clearly |/| = |V|, and so (V,T) = ±\V\ \N h \. 
By (|4.1ip the vector V vanishes at most two times along 7. Suppose that sq £ I' is the 
first positive value where V(sq) = 0. Note that the sign of //|V| is constant along a small 
interval (s - 5,s ). By gU) and (I4T01) we get (V,T)(s ) = (V,T)'(s ) = (V,T)"(s ) = 0. 
By using L'Hopital's rule twice, we deduce 

±|iV,|(7( S o)) = hm = hm ^yyjy(-s) = Jm |y/|2 + <y> «• 

The numerator tends to zero since \ V\' = {V/\V\,V) < | V^'| ^ M on (sq—5, s ). The denom- 
inator goes to |V(so)| 2 , which is positive; otherwise, the Jacobi field V would be identically 
zero along 7. It follows that |JV) l |('y(»o)) = 0, a contradiction since j(sq) 6 E - £q. 

To prove (iii) let us suppose that (V,T^ never vanishes along 7. Then it is clear that 
v = |(yr)| 1 /2 = (— ^V, T)) 1 / 2 since V(0) = S p . In particular, we get / = \V\ > along 
7. Now we derive v = (— (V, T)) 1 / 2 = (/jA^I) 1 / 2 with respect to s. By taking into account 
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((461) and ([4~T0|) . we obtain 



i(-(^T)- 3/2 )(^r)' = -,- 3 <y,„) 



~(N,T) 



/i/a|jv fc |3/2' 




After simplifying, we conclude by (|4.4j) that 



z(0 3 - 5 Z(Z(„- 2 )) - |w»|-'<N, T) z(<r 2 ) 



which proves the claim. 



□ 



Proposition 4.6. Let E be a C 2 complete, oriented, area- stationary surface immersed in 
H 1 m£/i empty singular set. Then the operator C defined in (|3.44[) satisfies C{\Nh\) ^ on 
E. Moreover, C(\Nh\){p) — for a point and on/?/ «/ (TV, T\ = and (B(Z), 5) = 1 

along the characteristic line o/E passing through p. As a consequence, C(\Nh\) =0 onY. if 
and only if any connected component of E is a Euclidean vertical plane. 

Proof. Take a point p £ S. Let r : / — > E be the integral curve through p of the vector 
field S in (|2TU1) . We define the map F : I x R ->■ H 1 by F(s, s) := F(e) + s Z r(e) . By the 
completeness of S and Proposition 13.31 (i), any 7 £ (s) := F(e, s) is a characteristic curve of 
S. In particular, F(I x M) C E. 

Let y(s) := (dF/de)(0, s). By using Lemma l4~5l we deduce that, along the complete line 
7(s) := 7o(s), the vectors V(s) and 7(5) generate the tangent plane to E at 7(5). Since E 
has empty singular set, it follows that the function (V,T^(s) — as 2 + bs + c never vanishes 
along 7(s). In case a = we must have & = (otherwise we would find a root of as 2 + bs + c). 
In case a^Owe must have b 2 — 4ac < 0. Anyway, we get b 2 —Aac ^ and so £(\Nh\)(p) ^ 
by Lemma 14.51 (i) . 

Observe that C(\Nh\)(p) — if and only if a = b = 0. This is equivalent to that (V, T) is 



constant along 7. It follows from (|4TB) and (j4TTU|) that (iV,T) = and (B(Z), S) = 1 along 
7. Conversely, if (iV,T) = and <S(Z), 5) = 1 along 7 then implies that C(\N h \) = 



Finally, if £.(\N h \) = on E then (N,T) = on E. By [31 Prop. 6.16] we conclude that 
any connected component of E must be a Euclidean vertical plane. Conversely, it is not 



Now we are ready to prove the main result of this section. 

Theorem 4.7. Let E be a C 2 complete, oriented, connected, area- stationary surface im- 
mersed in H 1 with empty singular set. If E is not a Euclidean vertical plane then E is 
unstable. 



along 7. 



difficult to see that C(\Nh\) = holds for any Euclidean vertical plane. 



□ 
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Proof. Let N be the unit normal vector to E. We can find p e E such that (N,T)(p) ^ 0. 
Otherwise E would be a Euclidean vertical plane by [33l Prop. 6.16]. By using Proposi- 
tion [X31 (i) and the completeness of E, we can parameterize E, around the characteristic line 
containing p, by the map F:/xM->S given by F(e, s) = F(e) + s Z t m, where T(e) is a 
piece of the integral curve through p of the vector field S in (|2.10p . Let -f E (s) := F(e, s). By 
Lemma T4. 5 1 we know that V e (s) :— (dF/de)(e, s) is a non- vanishing Jacobi field orthogonal 
to 7 e (s). Moreover, the function (V^(s),T) is strictly negative since E has empty singular 
set and V £ (0) = S r(e) . We consider the function v(e, s) := \(y e (s),T)\ 1 / 2 = [\N h \ IV^(s)I) 1 / 2 , 
which is continuous and C°° along any 7 £ (s). 

Now we use the coarea formula to compute the index form (|4.3p in terms of the coordinates 
(e, s). The Riemannian area element can be expressed as 

dE = \V e \ deals. 

Hence by using the definition of v together with Lemma 14.51 (iii) , equation (I4.3[) reads 
(4.12) l(uv- 1 \N h luv- 1 \N h \)= [ (^f) deals-- ( C{\N h \) u 2 de ds, 

for any u 6 Co(/ x R) which is also C 1 with respect to s. 

Take a non-negative C°° function <j> : I M with </>(0) > and compact support con- 
tained inside a bounded interval /' C I. Denote £ := length(I'). Let M be a positive 
constant so that M, eg/. For any k G N we define the function 

u k {e,s) := (j>{e)cj){s/k). 

It is clear that Uk £ Co(I' x kl'), and that Uk is C°° with respect to s. By Fubini's theorem 
2 1 / r \ / r \ cn/r2 



f^±Y dsds= _L( f ftefde) ( I ^(s/k) 2 ds) / 0(e) 2 de, 



Ixl 



\ ds J k 2 \J p J \Jki, J k 



which goes to when k — ¥ oo. Note also that {itfcjfceN pointwise converges when k — > oo 
to u(e, s) = <fi(0) (j)(e). By Proposition 14.61 we have C(\Nh\) > on E. Thus we can apply 
Fatou's lemma to obtain 



liminf / £{\N h \)u 2 k deds ^ £{\N h \) u 2 de ds. 

k^co J IxR JlxR 

We conclude from (|4.12p that 

limsup I(ukV~ l \Nh\, UkV~ 1 \Ni l \) = — liminf f £(\Nh\) u\de ds 

^ -7 / C{\N h \)u 2 deds, 

which is strictly negative by Proposition 14.61 since (N, 7^ inside an open neighborhood 
around p. Hence E is unstable. □ 

Corollary 4.8. Let E be a C 2 complete, oriented, connected, area- stationary surface im- 
mersed in H 1 with empty singular set. Then E is stable if and only if E is a Euclidean 
vertical plane. 



Proof. The necessary condition follows from Theorem 14.71 Conversely, suppose that E is a 
vertical plane. We can prove that E is an area-minimizing surface in H 1 by using a calibra- 
tion argument similar to the one in [531 Thm. 5.3], see also [H Ex. 2.2]. In particular, E is 
stable. □ 
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Remark 4.9. Previous results related to Corollary 14.81 were obtained in [4] and [15J . Pre- 
cisely, in [H Thm. 5.1] it is proved that the Euclidean vertical planes are the only complete 
stable intrinsic graphs in H 1 associated to a C 2 function. In 15, Thm. 1.8] vertical planes are 
characterized as the unique complete stable C 2 Euclidean graphs with empty singular set. 
As we pointed out in the introduction of the paper, Corollary 14.81 does not follow from the 
aforementioned results. For example, they do not apply for the family of sub-Riemannian 
catenoids t 2 = A 2 {x 2 + y 2 - A 2 ), A ^ 0. 

5. Complete stable surfaces with non-empty singular set 

In this section we give the classification of C 2 complete stable surfaces in H 1 with non- 
empty singular set. By Proposition 13.31 the singular set of a C 2 area-stationary surface 
consists of isolated points and curves of class C 1 . Moreover, the characteristic curves in 
the regular set meet the singular curves orthogonally. By using these facts we were able to 
obtain the following result in [33J Thm. 6.15]. 

Proposition 5.1. Let E be a C 2 complete, oriented, connected, area- stationary surface 
immersed in H 1 with singular set So- 

(i) //So contains an isolated point then E coincides with a Euclidean non-vertical plane. 

(ii) //Eo contains a singular curve then E is either congruent to the hyperbolic paraboloid 
t = xy or to one of the helicoidal surfaces Hr defined below. 

In [33J Ex. 6.14] we described the helicoid Hr as the union of all the horizontal straight 
lines orthogonal to the sub-Riemannian geodesic in H 1 obtained by the horizontal lift of the 
circle in the iry-plane of radius 1/R centered at the origin. We can parameterize Hr by 
means of the C°° diffeomorphism F : M 2 — > Hr defined by 



The singular set of Hr consists of the helices s = ±1/R. Note that the family {Hr}r>q 
is invariant under the dilations S\ defined in (|2.13p . In fact, it can be checked from (|5.1[) 
that 6x(Hr) = Hr/ with R' — e~ x R. The surfaces Hr coincide with the classical left- 
handed minimal helicoids in R 3 . In particular, they are embedded surfaces containing the 
vertical axis. We remark that the classical right-handed minimal helicoids in R 3 are com- 
plete area-stationary surfaces in H 1 with empty singular set, and so they are unstable by 
Theorem ET~71 

Proposition lS. II indicates us that the study of stable surfaces in H 1 with non-empty singu- 
lar set can be reduced to three cases: Euclidean non-vertical planes, the hyperboloid t — xy 
and the helicoids Hr. In [33J Thm. 5.3] we showed that any complete C 2 area-stationary 
graph over the xy-plane is an area-minimizing surface. This gives us the stability of any 
plane t = ax + by and any surface congruent to t — xy. So it remains to analyze the stability 
of the helicoidal surfaces Hr. 

We first compute some geometric terms of a helicoid Hr with respect to the system of 
coordinates (e, s) in (|5.1[) . Note that 



(5.1) 



F(e, s) — (s sin(i?e),s cos(Re) , e / R) . 



OF 

dF 
~ds~ 



s'm(Re) X + cos(ife) Y, 



Rs cos(i?e) X -Rs sin(Re) Y + f(s) T, 



where / : R —> R is defined by 



f(s) = --Rs 2 . 
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As a consequence, the Riemannian area element is given by 



(5.2) dE = y/f(s) 2 + R 2 s 2 de ds. 

On the other hand, the cross product of dF/ds and dF/de in (H 1 , g) provides the following 
unit normal vector to T-Lr 

fjs) cos(A £ ) X - f(s) sin(Jte) Y-RsT 

and so 

( 5 - 4 ) 1^1= 7^=ft^ <^ T >- 



^/(s) 2 + i? 2 s 2 x ' ^/(s) 2 + A> 2 s 2 
It follows that the straight lines 7 e (s) = F(e, s), s G R, satisfy 

(5.5) %(s) = S ign(l/R-\s\)Z, \s\^l/R. 
By taking into account (|3.29[) and (|5 .4|) we get, for \s\ ^ 1/R, that 

(5.6) (B(Z), S) = l+ \N h r Z((N, T» = ^ = - 1, 
which in particular implies 

(5.7) \B(Z) + S\ 2 -W 2 - ( R2 -^ ! ^ 2 



(f(s) 2 + R 2 s 2 



Now we are ready to deduce from Theorem 13.71 and Proposition 13.111 a stability criterion 
for helicoidal surfaces that plays the same role as Proposition ^. 121 

Proposition 5.2. Let E be the helicoid T-Lr. 7/E is stable then, for any function u G Cq(E) 
such that Z(u) = inside a small tubular neighborhood of Eo, we have Q(u) ^ 0, where 

(5.8) G(u) := / \Nh\~ 1 {Z{uf - (\B(Z) + S\ 2 - A\N h \ 2 ) u 2 } dE 

JT, 

-4 / u 2 dE + / S{ufdZ - 



Here {Z, S} is the orthonormal basis in (|2.9p and (|2.10p , i? is £/ie Riemannian shape oper- 
ator o/E, and dEo is i/ie Riemannian length measure on Eo- 

Proof. We suppose that the unit normal N to E is the one in (|5.3[) . For simplicity we de- 
note g = \B{Z) + S\ 2 - A\N h \ 2 . By IpT?) . (ETHl and (JSHJ) it follows that (A^-V 2 G L X (E) 
provided it G Co(E). In particular, Q(it) is well defined for any it G Co(E) which is piecewise 
C 1 in the Z-direction, satisfies \Nh\~ 1 Z (u) 2 G L 1 (E), and whose restriction to Eo is C 1 . 

Let us show, in a first step, the following statement 

(5.9) Q(v) > 0, for any v G C 2 (E) such that Z{v/(N, T)) = 

in a small tubular neighborhood U of Eo- 

Note that a function v as above satisfies Z(t>) 2 = (Z((N, T)) 2 /(N, T) 2 ) v 2 in E. It follows 
from (ED) and JO} that | A^ ft J -1 ^) 2 G L 1 (E), and so Q{v) < oo. 

Let cto be the radius of A and A the support of v. For any cr G (0,ao/2) let A CT be the 
tubular neighborhood of Eo of radius a. We consider functions h a ,g a G Cq°(E) such that 
g a = 1 on A' n A CT , supp(<7 ff ) C A 2(T and h a + g a — 1 on A". We define the C 2 vector field 

U a := (h r7 v)N + g a j^T, 
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whose support is contained in K. Note that {U a , N) — v on K. Let (fr(p) := exp p (r(U a ) p ) 
be the variation associated to U a and A a (r) :— A((p^(S)) the corresponding area functional. 
The variation ip* is vertical when restricted to E a . Hence we can suppose, by applying 
Proposition 13.111 to w — v/(N,T\, that the second derivative of A\ a {r) := A((p°(E a )) is 
given by 

a'Uo)= I s{vf<m . 



In the previous equality we have used that (N,T\ = ±1 on So- On the other hand, the 
second derivative of Ai,j(r) := A((p^(E — E a )) can be computed from Theorem 13.71 We 
obtain the following expression 

4U0)=/ \Nh\- 1 {Z(v) 2 -qv 2 }dZ+ f div s (£Z)dE + f div E (/iZ)dE, 
where 

t = (N,T)(l-(B(Z),S))v 2 , 

» = W j^.T) (1 - (B(Z),S)) J^jL 2(B(Z),S) . 

If E is stable then A"(0) > 0. As A a {r) = A\ a (r) + A 2a (r) we deduce, by using the classical 
Riemannian divergence theorem that, for any a 6 (0,<7o/2), we have the inequality 

(5.10) f \Nh\- 1 {Z{vf -qv 2 }dZ- [ + ti){Z, V )dl+ [ S{v) 2 dZ > 0, 

where rj is the unit normal to dE„ pointing into E — E a and dl denotes the Riemannian 
length element. 

Let us compute the boundary term above. Fix k 6 {1,2}. Let A be one of the two 
components of dE a at distance a of the singular curve where (A, T) = (— l) k+1 . By taking 
into account (|5-5[) it follows that r\ = (—l) k+1 Z along A. Moreover, the functions £ and fj, 
are constant along A. Since g a = 1 and h a — on A we have 



(5.11) / £ (Z, r,) dA = (-l) k+1 (A, T) (1 - (B(Z), S)) / v 2 dA, 

J A J A 

(5.12) / n(Z,v)dA=(-l) k + 1 \N h \ 2 (N,Ty 1 (l-(B(Z),S)) [ v 2 dA. 



Now we let u — > in (|5.10[) . From the dominated convergence theorem we get 
lim / \Nh\~ 1 {Z{vf -qv 2 }dll= f \Nh\- 1 {Z(v} 2 -qv 2 }dZ, 

Js-Ev JT, 

lim / v 2 dl = 2 v 2 dE . 

On the other hand, equation (|5.6|) yields (B(Z), S) — > —1 when we approach Eq. Moreover, 
we know that \N h \ -» and (A, T) — > ±1 when a — > 0. This facts, together with (|5.1ip and 
(|5~T2")l imply that 

lim [ (£ + fi) (Z, ji) dl = 4 / v 2 dE . 

JdE„ Jt, 

Hence we obtain Q(v) ^ from (|5.10[) . This proves (|5Ti 



Now we take u G C 2 (E) with Z(u) = inside a small tubular neighborhood E of Eq. For 
any a £ (0, 1) let be the open neig hborhood of E such that \(N,T)\ = 1 — a on dD a . 
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We can find cxo > such that D a C E for a e (0, Co). For such values of a we define the 
function <j) a : E — > [0, 1] given by 

in D a , 
in E — D a . 

Clearly </) CT is continuous and piecewise C 1 in the Z-direction. Moreover, the sequence 
{<t ) a}a^(Q,a ) pointwise converges to 1 when a — > 0. By using (|5.4I) and (|5.2p we can see that 
\Nh\~ 1 Z((N, T)) 2 extends to a continuous function on S, and so 

lim / \Nh\~ 1 Z(</v) 2 dE = 0. 

By a standard approximation argument we can slightly modify <p a around dD a in order 
to construct a sequence of C 2 functions {Vv}o-e(o.<T ) satisfying the same properties. De- 
fine v a :— ^(jU. This provides a sequence of functions in Cg(E) such that v„ = u in Eo 
and Z(v a /(N,T}) = inside a small tubular neighborhood of Eo. As a consequence of 
(|5.9p we have Q(v a ) for any er G (0, (To)- Finally, it is straightforward to check by us- 
ing the dominated convergence theorem and the Cauchy-Schwartz inequality in L 2 (T,) that 
{Q(v a )} — > Q(u) when a — > 0. The proposition is proved. □ 

Remark 5.3. By using Proposition 13.111 inequality Q(u) ^ can be generalized for any 
C 2 stable solution E of the Plateau problem whose singular curves are of class C 3 whenever 
supp(it) is contained in the interior of E and Z(u) = inside a tubular neighborhood of the 
singular curves. 



Now we are ready to prove the main result of this section. 
Theorem 5.4. The helicoidal surfaces Wr are all unstable. 



Proof. To prove the claim it suffices to show that H2 is unstable. In fact, for any R > we 
have Hr = <5a(%2), where d\ is the dilation defined in (|2. 13(1 with A = log(2/i?). By virtue 
of Lemma I3T21 we deduce that Hr is stable if and only if H 2 is stable. 

Let E :— % 2 - Consider the diffeomorphism F : R 2 — > E in (|5.1[) . We denote %(s) = 
F(e, s), sGR. The singular set Eo consists of the singular curves F(e, —1/2) and F(e, 1/2), 
E6l. We suppose that the normal N to E is the one in (|5.3p . By equation (|5.7I) we get 

\B(Z) + S\ 2 - 4\N h \ 2 = 0, 011E-E0. 

In particular, the quadratic form Q in (|5.8|) is given by 

(5.13) Q(u) = f \N h \- x Z(u) 2 dE - 4 j w 2 dE +/" S(u) 2 dE , 

for any u G Co(S) which is piecewise C 1 in the Z-direction, satisfies lA^I" 1 ^) 2 e £*(£), 
and whose restriction to Eo is C 1 . We apply in (|5.13p the coarea formula. By using (|5.4p , 
(|5.5p and (|5.2p . we deduce that 

(5.14) Q(u)= f H f f ^ s2 (^) dsds-4 f u(e,-l/2) 2 de-4 f u(e,l/2) 2 de 

+ / R (| U(£ '- 1/2) ) 2& + / R (^ (£ ' 1/2) ) 2,i£ - 

Let 4> '■ K — > M. be any C°° function with compact support [— £0, £q]- For any k > 1/2 and 
S > 0, let <pk8 : M — > [0, 1] be the symmetric function with respect to the origin given, for 
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s > 0, by 

{ 0, s^k + 5. 

Now we define the function Uks on E whose expression in coordinates (e, s) is 

u kS (s,s) = (j)(e)4>ks{s). 

Clearly u^s is a function in Cb(E) which is also C°° with respect to e and piecewise C°° in 
the Z-direction. Note also that 

(5.15) u w (e,-l/2) = « w (e,l/2) = ^(e), eei 

Moreover (diiks/ds)(s, s) = 4>{e) 4>' kS {s), which vanishes if \s\ < k or \s\ > k + S, and equals 
±</>(e)/<5 if k < s < k + S. This implies that Z(uks) = inside a tubular neighborhood of 
So- By using Fubini's theorem and that |/(s)| _1 (/(s) 2 + 4s 2 ) is symmetric with respect to 
the origin, we have 

(5 , 6) / (r *r*) fi r . 



i/wi v «« ; V-„ ' J \s 2 Jk i/w 

The second integral in the right-hand side can be easily computed. We obtain 

f k+S /(s) 2 +4s 2 f k+s 16s 4 + 8s 2 + 1 4s 3 /2s-l\l fc+5 
2 / v f— ,, ds = / - ds = h 3s + log 

A 1/001 A 4 S 2-i 3 H2.s + iyJ fe 

By an elementary analysis we can find a value k > 1/2 and 5 = 2A; + 1 such that the integral 
above times 1/S 2 is strictly less than 8. By substituting this information into (|5.16p . and 
using (15.141) together with (15.15[) . we conclude for v :— Uks 



Q(v) <M { ° c/)(e) 2 de + 2 f ° (/.'(e) 2 de 



for some constant M < which does not depend on the function <j>. If Eq is large enough, 
then we can choose <j) with compact support [— eo,eo] such that the right-hand side of the 
previous equation is strictly negative. This can be done since 




0'(e) 2 de) (J 0(e) 2 de) ; e C^°(R) 1 = 0. 



Denote := ^fc^ for the particular values of k and 8 found above. We mollify <f> in order 
to obtain a sequence of functions v a (e, s) — 0(e)</> CT (s) in C^°(S) with v a — v on Eq and 



lim / \N h \- 1 Z{v a ) 2 d^= [ {Nh^ZivfdX. 



Hence we have 



lim Q(v a ) = Q(v) < 0. 

(X — ^0 



By Proposition 15.21 we conclude that E is unstable. □ 

Remark 5.5. Though the helicoids T-Lr are unstable, it is possible to obtain by means of 
a calibration argument similar to the one used for the hyperboloid t = xy in [331 Thm. 5.3] 
that the surface obtained by removing the vertical axis from 1-Lr is area-minimizing. On 
the other hand, the second derivative of the area in Theorem 13 . 71 indicates us that any non- 
singular variation induced by a vector field U — vN + wT such that v and w are C 1 functions 
whose support is contained in the regular set of 7^2 satisfies A"(0) ^ 0. This means that H2 
is also stable under the variations used in Theorem 13.71 The proof of Theorem 15.41 shows 
that, to get that Hi is unstable, we need to consider a function whose support intersects a 
large piece of H2 containing the vertical axis and the singular set. 
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6. Main result 

As a consequence of our previous stability results we can prove the following. 

Theorem 6.1. Let £ be a C 2 complete, oriented, connected, area- stationary surface im- 
mersed in H 1 . Then £ is stable if and only if £ is a Euclidean plane or £ is congruent to 
the hyperbolic paraboloid t = xy. In particular, £ is area-minimizing. 

Proof. If £ is stable and the singular set £o is empty then £ must be a vertical plane by 
Theorem 14 .71 If £ is stable and £o ^ then Proposition 15.11 and Theorem 15 .41 imply that £ 
coincides with a non- vertical Euclidean plane, or it is congruent to the hyperbolic paraboloid 
t — xy. That Euclidean planes and surfaces congruent to t = xy are area-minimizing follows 
from [4, Ex. 2.2] and [33 Thm. 5.3]. □ 
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